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In analogy to Brownian computers we explicitly show how to construct stochastic models, which 
mimic the behaviour of a general purpose computer (a Turing machine). Our models are dis¬ 
crete state systems obeying a Markovian master equation, which are logically reversible and have 
a well-defined and consistent thermodynamic interpretation. The resulting master equation, which 
describes a simple one-step process on an enormously large state space, allows us to thoroughly in¬ 
vestigate the thermodynamics of computation for this situation. Especially, in the stationary regime 
we can well approximate the master equation by a simple Fokker-Planck equation in one dimension. 

We then show that the entropy production rate at steady state can be made arbitrarily small, but 
the total (integrated) entropy production is finite and grows logarithmically with the number of 
computational steps. 


PACS numbers: 05.70.-a, 05.70.Ln, 05.40.-a, 89.70.-a 

I. INTRODUCTION 

Computers are physical systems and information has 
to be stored and transmitted using physical devices. This 
trivially sounding statement has led to very important in¬ 
sights as soon as one starts to ask for the fundamental 
physical limits of computation. The most known state¬ 
ment is probably Landauer’s principle: erasing a data 
set with information content (i.e., entropy) H causes a 
minimum heat dissipation of I3~^H if /? is the inverse 
temperature of the surrounding environment. This was 
first formulated by Landauer in 1961 [1]. More generally, 
it was argued by Bennett [2-4] and others [5-9], that 
each logically irreversible operation (as, e.g., information 
erasure) must be accompanied with a corresponding heat 
dissipation whereas each logically reversible operation can 
be implemented - at least in principle - in an energeti¬ 
cally neutral way. The heat flow of such computers, if op¬ 
erated slowly enough, is then equal to the change in their 
Shannon entropy (times j3~^) demonstrating the useful¬ 
ness of the Shannon entropy to describe thermodynamic 
processes. 

Although the question about the relation between 
logical and thermodynamical reversibility sounds like a 
purely academic question, it is indeed of practical impor¬ 
tance because one of the limitations of todays computers 
lies in the heat which they produce during computation 
and which is hard to be drained off quickly enough. In 
addition, it is well-known that the thermodynamics of 
computation can be successfully applied to resolve the 
famous Maxwell demon paradox [3, 4, 9]. 

Today, it seems that most physicists accept the ther¬ 
modynamics of computation as a well established field 
and also Feynman notes (page 160 in [8]): “I see nothing 
wrong with his [Bennett’s] arguments. [...] I concluded 
that there was no minimum energy [consumption of com¬ 
puters].” Indeed, however, criticism was raised against 
Bennett’s exorcism of Maxwell’s demon and the thermo¬ 
dynamics of computation already some time ago [10-12], 
which was subsequently defended by Bub and Bennett 
again [13, 14], but criticism and controversies still pre¬ 


vail for different reasons, mainly (but not only) on the 
philosophical side [15-26]. 

An important class of physical models used to illustrate 
the thermodynamics of computation are inspired by bio¬ 
chemical processes as, e.g., DNA replication [2, 3, 7, 8]. 
Indeed, copying a DNA strand can be regarded as a sim¬ 
ple computational task where the DNA strand represents 
a certain input signal, which is manipulated by enzymes 
to produce an identical copy of the input. Energetic bar¬ 
riers in the computational path, i.e., barriers between two 
logical states of the computation, can be overcome by the 
random thermal motion of the molecules involved and a 
bias in chemical potentials can be used to drive the com¬ 
putation in a desired direction. For a small enough chem¬ 
ical bias the average dissipation of energy per step can 
be made arbitrarily small and thus, one usually concludes 
that computation can be carried out thermodynamically 
reversibly. 

In a more general frame, systems which use the random 
thermal motion of its components to perform a compu¬ 
tation are usually called Brownian computers. However, 
in addition to the arguments presented above, a detailed 
and general mathematical treatment of such computers 
seems to be missing in the literature. Indeed, the authors 
of Refs. [2, 3, 7, 8] based their reasoning largely on inge¬ 
nious arguments instead of detailed calculations. If one 
finds more detailed (yet not very general calculations) in 
the literature [21, 26], then they seem to contradict the 
statement that Brownian computers are thermodynami¬ 
cally reversible. 

In the present contribution we will therefore treat the 
subject of Brownian computers in general, i.e., without 
having any specific computational task or physical prob¬ 
lem in mind. We will start by considering an arbitrary 
Turing machine (TM), which is known to be a model for 
a general purpose computer. This means that for each 
computable function (or algorithm) there exists a TM 
which can implement it. Moreover, it is even possible 
to construct a special TM, called a universal TM, which 
is able to simulate any other TM and hence, TMs are 
said to be computationally universal (an introduction to 
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TMs can be found in Refs. [8, 27]). Based on the ideas of 
Bennett we will then show how to construct a logically 
reversible TM and in addition, we show how to model 
this TM by a continuous-time Markov process, i.e., by a 
Markovian master equation (ME). The resulting model 
is then able to compute in a stationary regime where 
it transforms a string of incoming symbols (the inputs of 
the computation) into a corresponding string of outgoing 
symbols (simply called the outputs). 

The big advantage of a description in terms of a ME 
is that its thermodynamic behaviour is well understood 
since many years [28, 29], and also their stochastic be¬ 
haviour can be treated within a consistent thermody¬ 
namic formalism, which is known as stochastic thermo¬ 
dynamics [30]. Indeed, there has been a large interest 
recently in using small autonomous machines describable 
by, e.g., a Markovian ME, to address questions of infor¬ 
mation processing as, e.g., sensing, feedback or adapta¬ 
tion, in a thermodynamic context. Such machines were 
studied for abstract models [31-36], more general set¬ 
tings [37-41], in a biochemical context [42-46] or in the 
field of artificial nanostructures [47, 48]. Furthermore, we 
have reached the realm where experiments are performed 
at the Landauer limit [49-53]. However, to the best of 
our knowledge, there is no reference which has treated the 
question of how to describe a general computer within a 
ME framework and which has worked out its thermody¬ 
namic consequences in detail. We will find out that the 
entropy production rate of a Brownian computer can be 
made arbitrarily small while the total entropy production 
grows logarithmically with the number of computational 
steps, thereby resolving a part of the controversy between 
Bennett, Norton and others. 

Outline: Because the treatment of TMs is no standard 
subject taught in physics, we will give an introduction to 
it in Sec. II to make the paper as self-contained as pos¬ 
sible. Then, in Sec. Ill we will first show how to build a 
TM in a logically reversible way before we demonstrate 
how to map it to a ME. Because the details of a logical 
reversible TM are a bit technical but not of major im¬ 
portance for a general understanding of the rest of the 
paper, we will shift them to appendix A. Finally, after 
we have discussed the structure of the ME in Sec. Ill, we 
discuss the thermodynamics of Brownian computation in 
Sec. IV. A last section is then devoted to a summary of 
our results and an outlook on interesting future work. 


II. TURING MACHINES 

A TM T is an idealized machine to model or simulate 
problems in computer science (see Fig. 1). In the stan¬ 
dard treatment T has two parts: hrst, we have the ma¬ 
chine itself, which can be in some state q G Q where Q de¬ 
notes the finite set of internal machine states. Second, the 
machine has access to an external storage medium (usu¬ 
ally called the tape), which is divided into equal squares 
and each square contains either a special symbol s G S 



square 

FIG. 1: Simple sketch of a TM: the machine, specified by a 
state q, has access to an infinite tape, which is divided into 
equal squares. The machine scans with its head always only 
one square on the tape, on which it finds a symbol s (which 
might be also the blank symbol b) written. 


(where S is again a finite set) or it contains a blank b. 
The machine has a head with which it is coupled to one 
and only one square of the tape at each point of time. 

At each time step, the machine in state q reads the 
symbol s written on the square and subsequently it 
changes its own state to q', writes a new symbol s' on 
the square (which can be the same as the old one) and 
either shifts the tape one square to the left or to the right 
or stays where it is. Mathematically, these rules can be 
defined by three functions F, G and H: 

q' = F{q,s), 

s' = G{q,s), (1) 

d' = H{q,s) 

where d' G {—1,0,-|-1} encodes the direction of move¬ 
ment of the tape with —1, 0, -|-1 meaning move the tape 
right, do not move it, move it left. These three maps (to¬ 
gether with an agreement about the initial state, see be¬ 
low) completely define the action of T. Often one writes 
these maps in terms of a so-called quintuple 

(q,s,F{q,s),G{q,s),H{q,s)) = {q, s,q', s',d') (2) 

or simply as 

{q,s) ^ {q,s',d'). (3) 

A computation of T is then defined as follows: the ma¬ 
chine starts initially in a special state R S Q (we use 
R for “ready”) and scans by convention the first blank 
symbol to the left of a finite number of input symbols 
Sin = (si, S 2 ,..., sm) written on the tape. Note that we 
demand that there is no blank symbol in between the 
input symbols and that the input string is finite. Then, 
T starts to move right and reads the first input symbol 
si. It then proceeds according to the rules above [Eq. 
(1)]. After some time the TM might be done with the 
computation. We then assume that it shifts to the first 
blank symbol to the right of the remaining string of sym¬ 
bols Sout = (si, S 2 ,..., sm') written on the tape and then 
changes to a special final state H (H for “halt”). The 
string Sout is called the output or the result of the com¬ 
putation, which is again finite but not necessarily of the 
same length as the input (i.e., M' M is possible). In 
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short, we will also write a computation as T : Sin —>■ Sout 

or iF(Sin) = Sout- 

Hence, we see that the idea of a TM is very simple: 
given (T, s), i.e., a TM T with input s in standard format 
as above, it follows the rules (1) until it is done with 
the computation. Especially, we note that there is only 
a finite set of quintuples or rules (1) because Q and S 
were assumed to be finite sets. Introducing Nq = #Q 
and Ng = #5 (with denoting the cardinality of 
a set), we see that any TM is completely specihed by 
Nq{Ns + 1) many quintuples. Note that we also need a 
rule for the machine if it scans a blank symbol, hence the 
factor Ns + 1 = #(5 U {!>}). 

In view of these facts it seems very remarkable that, 
first of all, TMs are capable of universal computation 
and, second, that they can show an incredibly complex 
behaviour. The hrst property is related to the Church- 
Turing thesis - which has to be taken for granted though 
- which states that every intuitvely computable func¬ 
tion can be computed by a TM. The second property 
is reflected, for instance, in the fact that it is impossi¬ 
ble to design a TM Tjj, which tells us for an arbitrary 
given TM T and input s whether (T, s) will halt or not. 
This is the famous halting problem. Thus, it might be 
that the computation defined above never reaches the 
state H and goes on forever. In this case, the compu¬ 
tation simply has no result. Thus, the incredible power 
of TMs (namely their computational universality) has a 
serious drawback (namely their in general unpredictable 
behaviour). A much more detailed account of TMs can 
be found in Refs. [8, 27]. 


III. STOCHASTIC TURING MACHINES 
A. Setup and idea 

Examining the thermodynamics of computation can 
be done in many different ways. Here, as stressed in the 
introduction, we want to capture three main features: 
first, we want to look at a general computational prob¬ 
lem and not one specific task; second, we are interested 
in a logically reversible computer; and third, we want 
to model the computation stochastically, i.e., as a Brow¬ 
nian computer. The general thermodynamic picture is 
hence as sketched in Fig. 2. Some machine (with a very 
complex interior in general, see below) is coupled to a 
thermal reservoir at inverse temperature /3 and a work 
reservoir. The work reservoir can be used to drive the 
computation in a certain direction whereas the thermal 
reservoir is equipped with a well-dehned notion of heat 
and entropy. The task of our machine is to compute. It 
therefore receives input signals and transforms them to 
output signals corresponding to the result of the com¬ 
putation. Logical reversibility demands that we use two 
separate tapes for the inputs and outputs (henceforth 
called the input and output tapes, respectively). In fact, 
if we do not keep the initial inputs but simply overwrite 


heat bath 
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—new input 
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FIG. 2: Sketch of the general setup, which allows us to analyze 
any abstract computational process in terms of thermody¬ 
namic quantities. The upper tape corresponds to the output 
tape, which is initially blank, and the lower tape corresponds 
to the input tape. Note that we will always assume that blank 
tapes are for free, i.e., the machine can have as many blank 
space on which it can write as it requires. Furthermore, the 
machine itself has access to two additional internal tapes (not 
sketched), see Sec. IIIB. 


them with the output (as the TM from Sec. H would do), 
our machine will be in general irreversible [60]. 

More specifically, a logically reversible computer is in 
principle able to unambiguously retrace its computational 
path (i.e., the sequence of logical states visited so far) 
back to the initial state. In fact, most TMs as intro¬ 
duced in Sec. H are logically irreversible, for instance, 
already due to the fact that the machines usually do not 
remember from which direction they were coming from. 
But even if it would remember this (for instance by writ¬ 
ing the direction d on an additional tape), the machine 
might still be logically irreversible. Consider for exam¬ 
ple that there exists a pair of states and symbols (gi, si) 
and (<72,52) such that q' = E(gi,si) = F{q 2 ,S 2 ), s' = 
G{qi,Si) = G{q 2 ,S 2 ) and d' = H{qi,si) = H{q 2 ,S 2 ). 
Then, given the state {q', s') [or even {q', s', d')], the ma¬ 
chine has two possible predecessor states and it cannot 
know from which it was coming. Hence, it is logically ir¬ 
reversible. This situation is sometimes called the merging 
of two computational paths, see Fig. 3. 

However, even a logically reversible computer still pro¬ 
ceeds deterministically step by step along its computa¬ 
tional path. This deterministic behaviour unambigously 
defines a computational direction, see Fig. 3 again (we 
remark that the computational direction does not neces¬ 
sarily coincide with the “physical” direction of the move¬ 
ment of the tape, which can - as we have seen - be 
shifted either way). In contrast, we also want to look at 
a stochastic machine, which makes random transitions in 
both directions, i.e., it is also allowed to jump back to a 
previous computational state. In order to assure that we 
can control the direction of computation on average, we 
demand that we can change the potential energy exter- 
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FIG. 3: A computational path is defined by the way the ma¬ 
chine proceeds from state to state through a high-dimensional 
state space (each state of the machine including the tapes is 
symbolized by a black circle). Here, we sketched two different 
paths (solid red and dashed blue lines). For standard ma¬ 
chines (which compute only from left to right) each path is 
uniquely determined by the input signal string and the initial 
machine state. For a logically irreversible machine, however, 
it might happen that two different paths merge at some point 
(as shown on top) making it impossible to find the inverse 
of a state in general (that is to say there is a unique way to 
go from left to right in the diagram but not from right to 
left). In contrast, this cannot happen for logically reversible 
machines (as shown in the middle), but this feature comes 
at the cost of introducing additional states and tapes making 
the state space even larger. Finally, whereas the standard 
logically reversible TM still proceeds deterministically from 
left to right, a stochastic TM jumps randomly according to 
some rates [e.g., Wn+i,n and Wn,n+i as used in Eqs. (6) and 
(8)] and hence, it can move both ways (bottom). 


nally, for instance, by using a work source or by adjusting 
chemical potentials appropriately. 

In the next section we will present the main ideas of 
how to obtain a logically reversible TM from an irre¬ 
versible TM as discussed in Sec. II sparing the mathe¬ 
matical details to the appendix. Then, given that we 
have a logically reversible TM, we will show how to as¬ 
sociate a ME to it in Sec. Ill C. 


B. Logically reversible Turing machine 

The procedure to make a computation of a TM logi¬ 
cally reversible was explained in detail in a famous pub¬ 
lication by Bennett [2]. He explicitly showed - given an 


arbitrary TM T as described by Eq. (1) - how to con¬ 
struct a machine R which is computationally equivalent 
to T but always logically reversible. To accomplish this, 
Bennett introduced a finite number of new machine states 
and two additional tapes, which record the previous com¬ 
putational steps taken. Furthermore, the computation is 
broken up into three stages (each stage in general consists 
of many single computational steps) and in total, the log¬ 
ically reversible TM needs approximately four times as 
many steps as the irreversible one [2]. Before we proceed, 
we remark that the construction given by Bennett is, of 
course, not unique (as he discusses as well) but seems to 
be very convenient. It is also noteworthy that TMs acting 
on n tapes are not more powerful (in terms of what they 
can compute) than a TM described by Eq. (1) because 
one can be mapped onto the other [27]. 

In addition to the treatment of Bennett, who con¬ 
sidered only a single computation T : Sin —>■ Sout, we 
want to construct a machine which continuously pro¬ 
cesses a stream of incoming input strings of the form 
(... 6, s[jj, 5,..., &, Sin, 6,...). Thus, we imagine an infi¬ 
nite input tape with different input strings Sin,s[jj,... 
separated by blank symbols to mark the beginning and 
the end of each single input string. The output tape 
then contains the results of the different computations, 
i.e., it looks like (...,6,s[,^j = T(s[„), 5, ...,&, Sont = 
T(sin), b ...). In this picture our machine resembles the 
devices from Refs. [32-34, 37, 40, 48] in which also an 
external tape is manipulated but mainly to extract work 
and not for computational purposes though. 

Our logically reversible TM thus will have in total 
four tapes and one computational cycle proceeds in five 
stages. The four tapes are called the input, working, 
history and output tape. Whereas the input and output 
tape are supplied externally (see Fig. 2), the working and 
history tape belong to the machine itself. By this we es¬ 
pecially want to emphasize that we require them to be 
blank at the end of the computation such that they are 
ready for usage again [61]. More specifically, one com¬ 
putational cycle consists of the following five stages (also 
see Table I): 

1) Copy input onto working tape: A new input arrives 
at the machine on the input tape and the machine 
copies this input onto its working tape leaving it 
there in standard format at the end of the first 
stage. 

2) Compute: In this stage the actual computation is 
performed which finally maps the input to the out¬ 
put. Furthermore, a history tape keeps track of the 
intermediate steps such that the computer would be 
able to retrace each step. 

3) Copy output to output tape: If the computation 
halts, the output on the working tape is copied to 
the output tape and the working tape is reset to its 
position as at the end of stage 2. 

4) Retrace computation: The computer retraces all its 
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stage 

input tape 

working tape 

history tape 

output tape 

short form, see Eq. (4) 

1 ) copy input onto working tape 

input_ 

- 

- 

- 

T^T4T^T2T\{sin, b, b, b) 

2 ) compute 

input 

input 

- 

- 

T^T4T^T2{Sia_, Sin, b, b) 

3) copy output to output tape 

input 

output 

history 

- 

T^T4T^(ss_ri, Sout, h, b) 

4) retrace computation 

input 

output 

history 

output 

T^T4,(^S[-n^ Sout? h, Sout) 

5) erase working tape 

input 

input 

- 

output 

^5(^111; Sin? b, Sout) 


input* 



output 

(Sin'*f? b, b, Sout) 


TABLE I: One computational cycle consists of five stages where each stage can consist of many steps. The first line shows the 
initial situation of the tapes. After the first stage, the tapes are shown as in the second line, which serves as the initial state 
for the second stage and so on. The underbar denotes the current position of the machine head on the respective tapes. Note 
that at each stage the machine works with two tapes only, whereas the other two remain unchanged. Furthermore, in the very 
last line we used a * to mark the input the computer has already processed, see appendix A for details. 


computation such that the output on the working 
tape becomes the input and the history is blank 
again. This stage is the inverse of stage 2. 


5) Erase working tape: We erase the working tape 
with the help of the input tape such that the work¬ 
ing tape is blank again. Note that this erasure step 
is logically reversible because we have an identical 
copy of the input on the input tape. Finally, we use 
an additional symbol (*) to mark that we already 
performed a computation for the current input and 
the machine moves on to the next input on the in¬ 
put string. 


Stage 2, 3 and 4 were already treated by Bennett in 
Ref. [2]. In addition, we require stages 1 and 5 because 
we want that our machine works continuously and not 
only once. The reader who is curious about the de¬ 
tails of each step is refered to appendix A. Otherwise, 
instead of one big machine T doing a computation in five 
stages, it might also help to imagine hve small machines 
Ti,..., T 5 . Our big machine T is then nothing else than 
a composition or concatenation of these small machines, 
i.e., T = Ts o T 4 o T 3 o T 2 o Tl = T^T^T^T 2 Ti (similar to 
the composition of different functions). The action of T 
on the state of the four tapes (in the order of the input, 
working, history and output tape, respectively) can be 


written as 

1) 15242322 ^ 1(81111 b, b, b) = T5T4T3T2(Sin, Sin, b, b), 

2) r5T4T3T2(Sin, Sin, b, b) = 252423 (Sin, Sout > h, b), 

3) r524T3(si„,Sout,h,b) = in; Sout; h, Sout ) ; 

4) T5T4(s in; Sout; h, Sout) — ^5 (s in; Sin; b, Sout)? 

5) 25 (^in; 8out ? b, Sout) — (Sin*? b, b, Sout)- 

(4) 

Eq. (4) can be regarded as a short form of Table I. Here, 
b denotes a blank tape and h denotes the history tape at 
the end of stage 2. Clearly, if all the machines Ti,..., T 5 
are logically reversible, then also T is. 

C. Stochastic Turing machine 

In this section we will show how to use a continuous¬ 
time Markov process to model any logically reversible 
TM, which we will simply call a stochastic TM. We start, 
however, by repeating what a ME is and what we need 
for a consistent thermodynamic interpretation. 

A continuous-time Markov process describing the dy¬ 
namics of a system X corresponds to a set of states X 
and an associated probability Px to be in a state x G X, 
which changes according to a Markovian first order dif¬ 
ferential equation called the ME [29]: 

^Pa:(2) = (5) 

x' 

Here, the rate matrix Wxy has real-valued entries and 
fulfills Wx^x' = 0 for all x' S A. This guaran- 
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tees that probability is conserved throughout the evo¬ 
lution, i.e., = 0 for all t. If we want to 

equip the ME (5) with a thermodynamic interpreta¬ 
tion [28], we have to associate to each state x an energy 
Ex G M. and the rate matrix has to additionally fulfill a 
property called local detailed balance, which states that 
\cd^x,x'l^x '= —ld{Ex — Ex') where /d is the inverse 
temperature of the environment to which the system X 
has contact. Note that this automatically implies that, 
if Wx,x' 0 , then also Wx',x 0 , i.e., for each transition 
x' —>■ X the reversed one x ^ x' must also exist. This 
framework can also be extended to more general situa¬ 
tions involving, e.g., multiple environments at different 
temperatures [28], but we do not need more than the 
things just mentioned. 

Now, associating a ME to a logically reversible deter¬ 
ministic TM is in fact very easy. All we have to do is to 
change the (unidirectional) deterministic updating rules 
from appendix A into (bidirectional) probabilistic tran¬ 
sition rules, i.e., we allow for transitions in the computa¬ 
tional forward direction as well as transitions which just 
undo the last computational step (backward direction). 

It is worth pointing out that the underlying state space 
X of the ME is in general infinite, but this is not neces¬ 
sarily related to the size of the input tape. Remember 
that - due to the halting problem - a computation might 
go on forever even if it only received a hnite input. In 
fact, even if the computation halts, there is no general 
way to give a reasonable estimate of the size of X in 
advance [27]. However, on the other hand, the struc¬ 
ture of the ME is very simple and this is related to the 
fact that we build the machine in a logically reversible 
way. In fact, each state x G X has only two adjacent 
states, namely its logical predecessor and its logical suc¬ 
cessor state. Hence, our ME describes a simple one-step 
or birth-and-death process [29] or equivalently, according 
to Schnakenberg [28], we could say that the topology of 
the underlying network is trivial. In fact, if there were 
any branchings or loops in the underlying network, the 
computation would not be logically reversible anymore 
because then a state could have multiple predecessors or 
successors. Hence, quite generally we can put the final 
ME into the form 

^Pn(0 — {^Xn+l,n Wn—l,n)Pn{t) 

Wn^n+lPn+l{t) Wn^n—lPn—lit). 


Here, of course, n € Z is a multi-index denoting the entire 
machine and tape configuration. 

Let us discuss the general structure of the ME a lit¬ 
tle further. First of all, it is important to note that only 
the current squares of the tapes can change stochastically 
whereas the rest of the tapes, which is not coupled to the 
machine, remains hxed. In fact, although there is an in¬ 
finite number of possible different states, not all states 
X G X are coupled with each other. Which states are 
coupled to each other is determined by the rules from ap¬ 
pendix A and by the input strings because they single 
out a unique computational path through the “labyrinth” 
of states in X. 

In addition, the number of transition rules is always 
finite and fixed as expressed in appendix A. This is 
true independently of the number of computations or the 
lengths of the input strings. Although there seem to be 
quite a lot of rules, note that they suffice to build a uni¬ 
versal logically reversible computer. Of course, things be¬ 
come much easier if we relax some of the requirements. 
Hence, in a more pictorial language we could say that 
the hardware (i.e., the set of transitions rules with the 
associated rates) of our machine remains hxed, but the 
software (i.e., the inputs determining the computational 
path) can change. 

Thus, if we focus only on the computation for one input 
string, i.e., on the stages 2 to 4, the full rate matrix W 
in Eq. (5) decomposes into blocks for each input string 
s, i.e., it has the form 


W = 


( 


sil 


IH(s2 ) 


W"(S3) 


(7) 


■■■/ 


and no transition between different blocks is allowed. 
Here, we labeled the different input strings 81 , 82 , 83 ,... 
in some canonical way and note that each block can be 
inhnitely large if the computation does not halt. For 
each Si the ME describes a simple one-step process and 
by rearranging the states appropriately we can write each 
block as a tridiagonal matrix 



\ 


VF(s.) = 


— Wn,n-l — Wn-2,n-l 

Wn.n-l 

0 


HA-1,n 

— HA+l.n — HA-l,n 

HA-1-1,n 


0 

HA,n-|-l 

— HA-|-2,n-|-l — HA,n-|-l 




( 8 ) 


V 
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where Wn+i,n {Wn-i,n) denotes the forward (backward) 
rate at step n in the ME (6). Hence, to conclude, al¬ 
though the state space X is extremely large, the rate 
matrix is also extremely sparse, i.e., it contains only a 
small number of non-zero elements (in relation to the 
total number of elements). 

Finally, we would need to associate a consistent en¬ 
ergy landscape to our system and the rate of forward 
and backward transitions would then need to obey local 
detailed balance, which fixes the temperature of the en¬ 
vironment. We will discuss this issue in the next section. 


IV. THERMODYNAMICS OF BROWNIAN 
COMPUTATION 

A. Energy landscapes 


ested in a steady state regime, i.e., the average dissi¬ 
pation per step should be independent of the number of 
computational steps performed so far. This demand rules 
out quadratic or exponential energy landscape. Second, 
we are also still faced with the halting problem. This 
implies that we cannot know in advance the number of 
computational steps we need for one computational cy¬ 
cle. Thus, associating any particularly shaped energy 
landscapes like a sine or a hill (as in [26]) is unfeasible 
because we do not know, for instance, how to choose a 
senseful period for the sine. 

Hence, we conclude: the only feasible energy landscape 
with which we can ensure to control the speed and di¬ 
rection of computation independently of the number of 
computational steps (which we cannot know in advance) 
and which is translationally invariant on the state space 
n S Z of the ME (6) is an on average linear landscape. 


So far we have shown that a stochastic, logically re¬ 
versible TM can be modeled by a simple one-step process 
as given by the ME (6). To interpret it thermodynami¬ 
cally we still need to associate a consistent energy land¬ 
scape to it, which we could control externally via a work 
source or, alternatively, a bias in chemical potentials as 
it would be the case for biochemical processes. 

For the sake of simplicity, we will choose below a lin¬ 
ear energy landscape along the computational path, i.e., 
the difference in energy between a logical state and its 
successor state is taken to be the constant e (i.e., for 
e > 0 the computation proceeds on average in the for¬ 
ward direction along a chain of states with decreasing 
energies). This choice is in agreement with the one usu¬ 
ally appearing in the literature [2, 3, 8, 21]. Before we 
proceed, however, we discuss and justify this choice in 
more detail. 

First of all, in Sec. IV B we will actually discuss the 
thermodynamics of our model on a coarse-grained level 
of description. That is to say we will be interested in 
the regime where the computer was running already for 
quite a long time such that the variance (n^) — (n)^ of 
the number of computational steps is large compared to 
unity where we defined (n^) = n^Pn (t) ■ In this pic¬ 
ture, e might denote just an average slope in the energy 
landscape, i.e., we explicitly allow for spatial irregulari¬ 
ties in the energy landscape as long as they are not too 
large. More specifially, if denotes the energy of state 
n according to the ME (6), we demand that 

I 

^ X/ ~ (^) 

n=-N 

holds for all n and for N of the order of the variance 
(n^) — (n)^ such that the energy landscape looks linear 
at the coarse-grained level. 

Second, it is worth pointing out that in fact - except 
for the spatially allowed irregularities - no other energy 
landscape seems to be feasible for a general purpose com¬ 
puter. The reason for this is twofold: first, we are inter- 


B. Effective Fokker-Planck equation 


Having agreed on the (on average) linear energy land¬ 
scape we will choose the transition rates in Eq. (6) as 
follows: 

IT „,„+1 = Wn+i,n = (10) 


Here, T is some rate setting the overall time-scale of our 
problem and we see that the rates fulfill local detailed 
balance, i.e., In]!!^,.^ y^] — E/^_|_x) — 

—/3e where /3e > 0 favors a computation in the forward 
direction. 

In the limit where the mean (n) and variance (n^) — 
(n) are large compared to one, we can approximate 
derivatives by 


^ f.\ Pn+i{t) - Pn-l{t) 

“- 2 -' 

^Pn(t) « Pn+l{t) - 2pn{t) +Pn-l{t). 


( 11 ) 


Then, the Fokker-Planck equation (FPE) corresponding 
to the ME (6) reads 


Id d 


-2sinh^ -f cosh^^ 
2 2 dn 


Pn{t). (12) 


This FPE describes the movement of an overdamped 
Brownian particle in a constant force field and with a 
constant diffusion coefficient with n G M. denoting the 
position of the particle. It even admits an explicit solu¬ 
tion: assuming that the machine has started initially at 
some fixed state n = 0, i.e., Pn(t = 0) = d(n), we obtain 


Pn{t) = 


y^dTrP cosh(/3e/2)t 

n — 2r sinh(/3e/2)t]^ 
4r cosh(/3e/2)t 


X exp < — 


( 13 ) 








C. Thermodynamic discussion 

Discussing the thermodynamic behaviour of Eq. (12) 
can be done using standard methods, see e.g. Ref. [54], 
We first of all compute the mean number of computa¬ 
tional steps, which is 


We now note that for e —>■ 0 the last term vanishes 
quadratically, i.e., the heat dissipated can be made ar¬ 
bitrarily small in this limit. The first term, however, is 
independent of e but vanishes for f —>■ oo. Hence, we have 

lim lim5i(t) = 0. (22) 

t —^oo e—^0 


(n)(t) = 2rfsinh ^ (14) 

and hence, the speed of computation becomes 

v= j^{n)(t) = 2Ts\n\Y^, (15) 

which - as expected - can be controlled by e. Especially, 
we see that we have ?; > 0 for e > 0 and vice versa. 

The variance of the distribution is 


(n^)(t) — (n)^(t) = 2rf cosh ^ (16) 

and grows linearly with time. Based on this we might ask 
the question when does the computation become approx¬ 
imately deterministic, i.e., when does the mean dominate 
the standard deviation? Calculating their ratio yields 


i){t) 




= ^/2rt tanh ^ sinh (17) 


If we want this quantity to be much larger than one, 
we obtain a condition for the minimum amount of time 
we have to wait until our machine computes almost in a 
deterministic fashion: 

t» (^2rtanh^sinh^^ ~ 


Here, we performed an expansion in ,de ^ 1 at the end. 
Thus, the closer we get to the reversible limit, i.e., the 
smaller j3e (see below), the longer we have to wait until 
the computer starts to work reliably. 

Furthermore, we can explicitly calculate the Shannon 
entropy of our distribution, which is 


H{t) = — y dnpn{t)h\pn{t) ~ ^47rertcosh^ 


(19) 

Using Eq. (14) we can also write the Shannon entropy 
as 





27re coth 



(n)(t) 


( 20 ) 


i.e., the Shannon entropy scales with the average number 
(n) of computational steps as S (t) ^ In (n). 

The rate at which entropy is produced is given by the 
change in Shannon entropy plus /3 times the heat flow 
dissipated into the environment [54]. Since the latter is 
simply ev, we can write for the entropy production rate 

+ Pev 

1 /3e 

=-h 2r/3esinh — > 0. 

2t 2 - 


Thus, a Brownian computer can work thermodynami¬ 
cally reversibly (i.e., with zero entropy production rate) 
in the steady state regime if the bias e is small enough. 
This would confirm the conclusions from Ref. [2, 3, 8]. 

However, we can also confirm Norton’s perspective on 
the matter [21, 26]. Starting initially at t = 0 we see that 
the total amount of entropy produced up to time t is 

AiS{t) = [ dt'Siit') = H(t) + pe{n){t) > 0. (23) 

Jo 

Thus, even for e = 0 the Shannon entropy still grows log¬ 
arithmically with the number of computational steps be¬ 
cause the probability distribution of the machine spreads 
over the available phase space similarly to the free ex¬ 
pansion of a one-molecule gas, which is a thermodynam¬ 
ically irreversible process. If we think in terms of ther¬ 
modynamic cycles instead of a thermodynamic machine, 
which works in the stationary regime, we would have to 
dissipate an amount of entropy proportional to In (n) to 
reset the Brownian computer to its initial zero entropy 
state. However, again, compared to the number of com¬ 
putational steps (n) taken, the ratio ln((n))/(n) becomes 
arbitrarily small for a large number of steps, i.e., for a 
long computation. 

Finally, we remark that a full treatment in terms of 
the ME (6) instead of the FPE (12) would provide very 
similar results. In fact, the first two cumulants (mean 
and variance) can be shown to coincide. 


V. SUMMARY AND OUTLOOK 

Let us summarize our main hndings and discuss possi¬ 
ble interesting open questions based on our findings. 

We have started with an arbitrary TM as a model for 
a general purpose computer, which is, however, in gen¬ 
eral logically irreversible and free from any thermody¬ 
namic interpretation. We then discussed how a compu¬ 
tation can be regarded as a thermodynamic process and 
we decided to investigate the thermodynamics of a logi¬ 
cally reversible stochastic computer, which simulates the 
TM considered at the beginning. We explained in detail 
how to obtain a logically reversible computer out of an 
irreversible one and then used bidirectional probabilis¬ 
tic transition rules instead of unidirectional determinis¬ 
tic rules to model the stochastic or random motion of our 
computer. 

Although the problem seems to be very complex, we 
have seen that the resulting ME has the very simple 
structure of a one-step process, which describes how the 
computer follows a well-defined path in a large state 
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space and eventually maps the input signals to output 
signals (the results of the computation). We have ar¬ 
gued that the only feasible energy landscape for such 
a computer is an approximately linear one, which not 
only simplified the thermodynamic discussion, but also 
allowed us to solve a corresponding FPE describing the 
drift and diffusion of the computer exactly. 

We have then seen that our stochastic computer can 
work thermodynamically reversibly, i.e., in a dissipation- 
free fashion, in a steady state regime and in this respect 
Feynman was indeed right with his initially quoted state¬ 
ment. However, just because the entropy production rate 
can become arbitrary small, this does not imply that the 
overall integrated entropy production is zero. Especially, 
if we think in terms of a computational cycle, in which we 
want to reset the computer to its initial state at the end, 
there is an unavoidable cost due to the increasing Shan¬ 
non entropy of the probability distribution during the 
computation. In fact, this additional cost is not inde¬ 
pendent of the number of computational steps but scales 
logarithmically with it and it seems that this effect has 
been only recognized by Norton so far [21]. Here, we 
have verified this result in a conceptually clean and gen¬ 
eral framework. 

Eurthermore, it is worth emphasizing that our com¬ 
puter works error-free at a finite entropy production. In 
fact, by construction our model does only allow for tem¬ 
porary errors in the computation (due to the fact that our 
stochastic machine can randomly hop back to its previ¬ 
ous state), but in the long run each temporary error is 
corrected by the next step in the computational forward 
direction and there are no other sources of errors allowed. 
Including errors (for instance, random bit flips or - to 
avoid the halting problem and an infinitely long compu¬ 
tation - one could decide to terminate the computation 
after a fixed number N^ax of steps) in our scheme and 
investigating the thermodynamic cost to correct them 
might be an interesting project for the future. 

Another interesting question is whether we can sense- 
fully assign a notion of efficiency to our computer. Erom 
a purely physical point of view the machine we have con¬ 


sidered is actually senseless because it describes only a 
simple (and never-ending) relaxation process. However, 
the machine is indeed “working”, i.e., doing something 
“useful” for us, because it tells us the answer to many 
questions. But how can we quantify the usefulness of 
our machine? Having a rigorous notion of a thermody¬ 
namic efficiency for a computer would allow us to study 
question of, e.g., efficiency at maximum power, which 
is an important question for the design of realistic ma¬ 
chines, see, e.g., [55-59]. In this context, one can also 
ask the question whether a logically reversible computer 
is really desirable or whether a logically irreversible com¬ 
puter might indeed be able to work at a fundamentally 
better efficiency. At the end, it seems that biochemi¬ 
cal processes in our body work very efficiently, but not 
necessarily in a logically reversible way. 

Finally, let us say a few words about the relation be¬ 
tween the present work and the devices investigated in 
Refs. [32-34, 37, 40, 48] (also see Feynman for a simple 
example of such a device who called them information- 
driven engines [8]). Indeed, as in our case, these 
information-driven engines are coupled to an external 
tape or “information reservoir”. This additional reser¬ 
voir can then be used to extract work from a single 
heat bath while simultaneously writing information on 
the tape (i.e., increasing its Shannon entropy). This pic¬ 
ture, however, does not carry over to our situation. In 
fact, the Shannon entropies of the incoming and outgoing 
tapes are equal because the input tape gets mapped to 
itself and the output tape is uniquely determined by the 
input. For a logically irreversible computer this does not 
need to be true as it can be already seen from the devices 
in Refs. [32-34, 37, 40, 48] where it was also shown that 
they can be used as an information eraser. 
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Appendix A: Logical reversible Turing machine 

We here provide the detailed rules for the machine be¬ 
haviour at each stage of the computation. These rules 
are, of course, not unique. However, because we are not 
primarily interested in the speed or efficiency of our ma¬ 
chine, but only in what it can do, possibly different im¬ 
plementations are unimportant for the present context. 
Furthermore, note that at each stage the machine is only 
manipulating two tapes whereas the other two tapes re¬ 
main fixed (see Table I). We will therefore use the nota¬ 
tion Sm,tn\ where denotes the internal machine 
state at stage i, Sm the symbol s printed on square m 
of the first tape of interest and the symbol t printed 
on square n of the second tape of interest. What are the 
tapes of interest will become clear in the treatment of 
each stage. Note that the notation is different from the 
one used by Bennett [2]. 


1. Stage 1) copy input onto working tape 

We want to copy the input on the input tape (first tape 
of interest) to the working tape (second tape of interest). 
The input is given in the form Sjn = (si,..., sm) and we 
assume that the machine scans initially the symbol at the 
far right (i.e., sm) (see also Table I). Furthermore, the 
working tape is by construction initially completely blank 
(that this is so can only be seen after the completion of 
all five stages, of course). The copy stage then proceeds 


as follows: 

[^0 : t [(^^ 5 (sM)n] 

[( 7 q \ (sM-l)m-l, ^n-l] 

[g^ \ (sM-l)m-l! (sM-l)n-l] 

['7^^ ('Sl)m-(M-l)j (Sl)n-(M-I)] 

^ [^0 Mj^n —m] 

t [H^ \ ^m—M: ^n— m] • 

(Al) 

Hence, we see that during the copy operation the machine 
changes between the two states and where the 
first is responsible for copying the symbol on the input 
tape to the working tape and the second is responsible 
for a shift of both tapes. This procedure goes on until 
it hits the first blank symbol to the left of the input 
string. The machine then changes to the “ready” state 
, which is the special initial state for the second stage. 
Note that - due to the fact that the copying procedure is 
unidirectional, i.e., the machine moves the tape always in 
the same direction - each map has a clear logical inverse. 
Furthermore, the positions m and n of the squares are in 
general arbitrary and can be chosen initially at will. 


2. Stage 2) compute 

This is the central stage of our computational cycle. 
If we would not bother about logical reversibility, this 
would be the only stage to execute. We thus have to 
explicitly think about how to make the map (1) logi¬ 
cally reversible. The idea is the following [2]: first, be¬ 
cause each standard (i.e., irreversible) TM is defined by 
its Nq{Ns + 1) many quintuples, we introduce a set Q 
with ffQ = Nq{Ns 1) of additional machine states 
q G Q. Then, to each map (g, s) —>■ (g', s', d!) we can 
associate a special state g^s, which uniquely labels each 
quintuple. Second, we will make use of an additional 
tape called the history tape, which remembers the past 
qqs such that the machine is able to uniquely retrace its 
computational path. 

Stage 2 is thus a bit more complicated. The corre- 


spending maps are 


interest). This goes as follows 
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[R^^\bn-M, bm] 


^ [^^( 2)55 ^m+l] 

n —M+1 1 (9r( 2)6) m+l] 


step £ 




W {i) ,bm+l+l] 

\ ^( 2 ) 

^ W 


ise+i)n'+d', ^^)m+i+i] 


[R^'^\bn'-,bJ^ ^ [qi\bn»,b^] 

(‘y\ 

[9l 5 (sM')n"-lj ^m-l] 

“t [^0 j{^M')n"-l,{sM')m-l] 

(A3) 


bo ) ('Sl)n"-M') (si)m-M'] 
( 2 ) 

^ [^1 j ^n"—M' —1 ; M' — 1 ] 


/ o \ 

step r' )■ (2) 7 (^ly)n" —1; ^m+i/+l] 

^ [H^ \ bn",{q^{2)^ )m+i'+l]- 

(A2) 

The most important step to understand is the one in the 
middle, which corresponds to the £’th computational step 
of the ordinary irreversible TM defined by (1). Initially, 

( 2 ) 

the machine is in state and scans the symbol si on 
the square n' of the working tape. The history tape con- 
tains the state q^,L , which uniquely labels the pre- 

vious computational step. Then, the machine changes 
its state to q^'^h , writes the symbol s« on the square 

Qi se 

according to the function s'^ = G{qf'\si) and shifts the 
history tape one square to the left, which contains a blank 
symbol. Finally, we write q ( 2 ) to the history tape and 

/ r 2 ^ / 2 ) 

change the machine state to g/ = F{q\ ,si). Further¬ 
more, we shift the working tape one square according to 
d! = ,st) such that the machine now scans the 

new symbol s^+i. Then, the whole procedure can start 
again where - in order that we are able to apply map (1) 
- we identify because the final state of the 

machine at the end of step t is the initial state for step 

£ + 1. 

The first and last two lines of Eq. (A2) then simply 
describe the initial and final steps of the computation. 
Initially, the machines starts in and then shifts the 
working tape to the left such that it reads the first symbol 
Si and starts in the state q\ . Finally, if the machine 
halts, it reaches the state H and stops at the first blank 
to the right of the output of the computation. 


and is very similar to (A2). Finally, however, to prepare 
the machine for the next stage, we want that it scans the 
output on the working tape at the very right again (at 
the moment it scans the blank on the working tape to 
the left of the output). To accomplish this we use two 
more machine states: 

( 2 ) 

[92 \ bn"-M'-l,bm-M'-l] 

/ Q 

bs q (Sl)n"-M', &m-M'-2] 

^br,(s2) n"-M' + l,bm-M'-3] (A4) 

—t bs \ {SM')n"-l,bm-2M'] 

bs I 7 ^m-2M'-l]- 

('31 

Here, is an intermediate state and its sole purpose is 
to indicate that the copying procedure is over and the ma¬ 
chine starts now only to shift the working tape without 

(3) 

changing it. The state q^ then actually accomplishes 
this task by shifting the working tape one step to the left 
while simultaneously shifting the output tape one step 
to the right until it reaches the first blank symbol to the 
right of the output on the working tape, which will indi¬ 
cate the start of stage 4. 


3. Stage 3) copy output to output tape 

We now want to copy the output Sout = (si,..., sm') 
of the computation from stage 2 from the working tape 
(first tape of interest) to the output tape (second tape of 


4. Stage 4) retrace computation 

In this stage we will basically apply the inverse of stage 
2 such that at the end the working tape contains the input 
again and the history tape is returned to its initial blank 
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state. This goes as follows: 

[Qs ; )m+^'] 

^ [ 9 ^( 2 )^ J (S[/)ti" —1, 

bS i^i 2 -l)n"-l, )m+i^-l] 

“t [Qi \{si+i)„>+d',{qg(2)^Jm+l+l] 
A^i)n',bm+£+l] 

[qi^\ (se)n', 

-M+l, (9R(2){,)m+l] 

^ [9|^(2) ^n—Mj &m+l] 

—> 6„_m, &m] • 


would be that we have printed the result of the compu¬ 
tation on the output tape. However, if this were true, we 
would be doomed to repeat the same computation again, 
while in fact we want to compute with the next input 
on the input string. To achieve this we add the following 
rules: 


(A5) 


[q^\bjn+l,bn+l] 


2 •) ^m +11 f'n+lj 


[qi^ (sm) m 5 bn+2\ 

[92 \ (sM-l)m-l, ^n+s] 


[92 \ {-^ 1 )^ 1 -M ybn+M+l] 
[92 \ bTn-M-l,bn+M+ 2 ] 
[93 \bTn-M-l,bn+M+2] 


(A7) 


[93 \ ,bn\ 

[ 9 o \ isM')m' ,bfi]. 


Note that we are using the superscript (4) on the internal 
machine states to explicitly distinguish them from the 
states of stage 2 indicating that we are truly in a different 
stage here. 


5. Stage 5) erase working tape 


The last step consists in erasing the content on the 
working tape such that it is blank again and ready 
for the next computation. Note that the “erasure” of 
the working tape does not actually erase any informa¬ 
tion because the working tape contains the same input 
Sin = (si, • ■ •, sm) as the input tape. As in stage 1 we 
choose the first tape of interest to be the input tape and 
the second tape of interest is the working tape. Then, we 
actually only have to apply the inverse of stage 1, i.e.. 


[R.^ m] ^ 



[9o —Mi^n— m] 

[9P\ (Sl)m-(M-I)) (Sl)n-(M-I)] 
[q0^\ (Sl)m-(M-I)) ^n-(M-l)] 


[9p\ (sm) m 5 (sm)™] 

^ [9o ;(^M)mi^n] 

[9l \ ^m-l-1, ^n-l-l]- 


(A6) 


If we would postulate the final transition rule 
[9i^\ ^m+i, ^n+i] -t [9o^\ {sM)m, bn] we would be exactly 
back at the initial state of stage 1 and the only change 


Here, we first of all marked the input with a * to 
indicate that we have done already a computation for 
that input. We then used an additional state q^ , which 
simply traverses the input string Si„ until it hits a blank 

(5) 

symbol. The machine then changes to the state q^ and 
goes further to the left until it hits the next non-blank 
symbol on the input string. This symbol then indicates 
the beginning of the next input such that - starting 
from stage 1 again - we can readily execute the next 
computational cycle. 


6. Summary 

Suppose that the irreversible TM from Sec. H has 
Nq many internal states. Ns -f 1 many different sym¬ 
bols (including the blank) on the tape and hence, it has 
Nq = Nq{Ns -f 1) many quintuples. Furthermore, sup¬ 
pose it was given an input of length M and produced an 
output of length M' after v computational steps in total. 

Then, our reversible machine has 2 -|- {Nq -f Nq) 
{Nq-\-Nq)-\-4: states from the first, second, ..., fifth stage 
of the computation, i.e., in total 2{Nq-\- Nq) 10 states. 
Furthermore, it needs 2M -f 2z2 -|- {2M' 1)2iy{2M -f 

2 x) = 4i/ -(- AM -f 2M' -I- 3 -I- a; many computational 
steps. Here, the x denotes the number of unknown blank 
symbols separating the current input from the next input 
on the input tape (see stage 5). Note that we need a; > 2 
such that there is enough space for the symbol * and 
to guarantee that all input strings (including potentially 
the symbol *) are separated by at least one blank symbol 
from eachother. 


